#position bias

'LLM-as-a-Judge systems show measurable biases and attack vulnerabilities; their agreement with humans is task-dependent. Practical evaluation favors trace-based outcome metrics, component-level tests, careful prompting, and ensembling for constrained tasks.'

TransEvalnia leverages prompting-based reasoning with large language models to provide detailed, human-aligned translation evaluations, outperforming traditional metrics on multiple language pairs.